Audio upmixer operable in prediction or non-prediction mode

ABSTRACT

The invention provides methods and devices for outputting a stereo audio signal having a left channel and a right channel. The apparatus includes a demultiplexer, decoder, and upmixer. The upmixer is configured operate either in a prediction mode or a non-prediction mode based on a parameter encoded in the audio bitstream.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/849,622, filed Dec. 20, 2017, which is a continuation ofU.S. patent application Ser. No. 14/793,297, filed Jul. 7, 2015, nowU.S. Pat. No. 9,892,736, which is a divisional of U.S. patentapplication Ser. No. 13/638,898, filed Oct. 1, 2012, now U.S. Pat. No.9,111,530, which is the US national stage of International PatentApplication No. PCT/EP2011/055350, filed Apr. 6, 2011, which claimspriority to U.S. Provisional Patent Application No. 61/322,458, filedApr. 9, 2010, each of which is hereby incorporated by reference in itsentirety.

TECHNICAL FIELD

The invention disclosed herein generally relates to stereo audio codingand more precisely to techniques for stereo coding using complexprediction in the frequency domain.

BACKGROUND OF THE INVENTION

Joint coding of the left (L) and right (R) channels of a stereo signalenables more efficient coding compared to independent coding of L and R.A common approach for joint stereo coding is mid/side (M/S) coding.Here, a mid (M) signal is formed by adding the L and R signals, e.g. theM signal may have the form

M=(L+R)/2

Also, a side (S) signal is formed by subtracting the two channels L andR, e.g., the S signal may have the form

S=(L−R)/2

In the case of M/S coding, the M and S signals are coded instead of theL and R signals.

In the MPEG (Moving Picture Experts Group) AAC (Advanced Audio Coding)standard (see standard document ISO/IEC 13818-7), L/R stereo coding andM/S stereo coding can be chosen in a time-variant and frequency-variantmanner. Thus, the stereo encoder can apply L/R coding for some frequencybands of the stereo signal, whereas M/S coding is used for encodingother frequency bands of the stereo signal (frequency variant).Moreover, the encoder can switch over time between L/R and M/S coding(time-variant). In MPEG AAC, the stereo encoding is carried out in thefrequency domain, more particularly the MDCT (modified discrete cosinetransform) domain. This allows choosing adaptively either L/R or M/Scoding in a frequency and also time variable manner.

Parametric stereo coding is a technique for efficiently coding a stereoaudio signal as a monaural signal plus a small amount of sideinformation for stereo parameters. It is part of the MPEG-4 Audiostandard (see standard document ISO/IEC 14496-3). The monaural signalcan be encoded using any audio coder. The stereo parameters can beembedded in the auxiliary part of the mono bit stream, thus achievingfull forward and backward compatibility. In the decoder, it is themonaural signal that is first decoded, after which the stereo signal isreconstructed with the aid of the stereo parameters. A decorrelatedversion of the decoded mono signal, which has zero cross correlationwith the mono signal, is generated by means of a decorrelator, e.g., anappropriate all-pass filter which may include one or more delay lines.Essentially, the decorrelated signal has the same spectral and temporalenergy distribution as the mono signal. The monaural signal togetherwith the decorrelated signal are input to the upmix process which iscontrolled by the stereo parameters and which reconstructs the stereosignal. For further information, see the paper “Low ComplexityParametric Stereo Coding in MPEG-4”, H. Purnhagen, Proc. of the 7th Int.Conference on Digital Audio Effects (DAFx'04), Naples, Italy, Oct. 5-8,2004, pages 163-168.

MPEG Surround (MPS; see ISO/IEC 23003-1 and the paper “MPEG Surround—TheISO/MPEG Standard for Efficient and Compatible Multi-Channel AudioCoding”, J. Herre et al., Audio Engineering Convention Paper 7084,122^(nd) Convention, May 5-8, 2007) allows combining the principles ofparametric stereo coding with residual coding, substituting thedecorrelated signal with a transmitted residual and hence improving theperceptual quality. Residual coding may be achieved by downmixing amulti-channel signal and, optionally, by extracting spatial cues. Duringthe process of downmixing, residual signals representing the errorsignal are computed and then encoded and transmitted. They may take theplace of the decorrelated signals in the decoder. In a hybrid approach,they may replace the decorrelated signals in certain frequency bands,preferably in relatively low bands.

According to the current MPEG Unified Speech and Audio Coding (USAC)system, of which two examples are shown in FIG. 1, the decoder comprisesa complex-valued quadrature mirror filter (QMF) bank located downstreamof the core decoder. The QMF representation obtained as the output ofthe filter bank is complex—thus oversampled by a factor two—and can bearranged as a downmix signal (or, equivalently, mid signal) M and aresidual signal D, to which an upmix matrix with complex entries isapplied. The L and R signals (in the QMF domain) are obtained as:

$\begin{bmatrix}L \\R\end{bmatrix} = {{g\begin{bmatrix}{1 - a} & 1 \\{1 + a} & {- 1}\end{bmatrix}}\begin{bmatrix}M \\D\end{bmatrix}}$

where g is a real-valued gain factor and a is a complex-valuedprediction coefficient. Preferably, α is chosen such that the energy ofthe residual signal D is minimized. The gain factor may be determined bynormalization, that is, to ensure that the power of the sum signal isequal to the sum of the powers of the left and right signals. The realand imaginary parts of each of the L and R signals are mutuallyredundant—in principle, each of them can be computed on the basis of theother—but are beneficial for enabling the subsequent application of aspectral band replication (SBR) decoder without audible aliasingartifacts occurring. The use of an oversampled signal representation mayalso, for similar reasons, be chosen with the aim of preventingartifacts connected with other time- or frequency-adaptive signalprocessing (not shown), such as the mono-to-stereo upmix. Inverse QMFfiltering is the last processing step in the decoder. It is noted thatthe band-limited QMF representation of the signal allows forband-limited residual techniques and “residual fill” techniques, whichmay be integrated into decoders of this type.

The above coding structure is well suited for low bit rates, typicallybelow 80 kb/s, but is not optimal for higher bit rates with respect tocomputational complexity. More precisely, at higher bitrates, the SBRtool is typically not utilized (as it would not improve codingefficiency). Then, in a decoder without a SBR stage, only thecomplex-valued upmix matrix motivates the presence of the QMF filterbank, which is computationally demanding and introduces a delay (at aframe length of 1024 samples, the QMF analysis/synthesis filter bankintroduces a delay of 961 samples). This clearly indicates a need for amore efficient coding structure.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide methods andapparatus for stereo coding that are computationally efficient also inthe high bitrate range.

The invention fulfils this object by providing a coder and decoder,coding and decoding methods and computer program products for coding anddecoding, respectively, as defined by the independent claims. Thedependent claims define embodiments of the invention.

In a first aspect, the invention provides a decoder system for providinga stereo signal by complex prediction stereo coding, the decoder systemcomprising:

an upmix adapted to generate the stereo signal based on firstfrequency-domain representations of a downmix signal (M) and a residualsignal (D), each of the first frequency-domain representationscomprising first spectral components representing spectral content ofthe corresponding signal expressed in a first subspace of amultidimensional space, the upmix stage comprising:

-   -   a module for computing a second frequency-domain representation        of the downmix signal based on the first frequency-domain        representation thereof, the second frequency-domain        representation comprising second spectral components        representing spectral content of the signal expressed in a        second subspace of the multidimensional space that includes a        portion of the multidimensional space not included in the first        subspace;    -   a weighted summer for computing a side signal (S) on the basis        of the first and second frequency-domain representations of the        downmix signal, the first frequency-domain representation of the        residual signal and a complex prediction coefficient (α) encoded        in the bit stream signal; and

a sum-and-difference stage for computing the stereo signal on the basisof the first frequency-domain representation of the downmix signal andthe side signal,

wherein the upmix stage is further operable in a pass-through mode, inwhich said downmix and residual signals are supplied to thesum-and-difference directly.

In a second aspect, the invention provides an encoder system forencoding a stereo signal by a bit stream signal by complex predictionstereo coding, including:

an estimator for estimating a complex prediction coefficient;

a coding stage operable to:

-   -   (a) transform the stereo signal into a frequency-domain        representation of a downmix and a residual signal, in a        relationship determined by the value of the complex prediction        coefficient;

and

a multiplexer for receiving output from the coding stage and theestimator and encoding this by said bit stream signal.

In a third and a fourth aspect of the invention, there are providedmethods for encoding a stereo signal into a bit stream and for decodinga bit stream into at least one stereo signal. The technical features ofeach method are analogous to those of the encoder system and the decodersystem, respectively. In a fifth and sixth aspect, the invention furtherprovides a computer program product containing instructions forexecuting each of the methods on a computer.

The invention benefits from the advantages of unified stereo coding inthe MPEG USAC system. These advantages are preserved also at higher bitrates, at which SBR is typically not utilized, without the significantincrease in computational complexity that would accompany a QMF-basedapproach. This is possible because the critically sampled MDCTtransform, which is the basis of the MPEG USAC transform coding system,can be used for complex prediction stereo coding as provided by theinvention, at least in cases where the code audio bandwidths of thedownmix and residual channels are the same and the upmix process doesnot include decorrelation. This means that an additional QMF transformis not required any longer. A representative implementation ofcomplex-prediction stereo coding in the QMF domain would actuallyincrease the number of operations per unit time significantly comparedto traditional L/R or M/S stereo. Thus, the coding apparatus accordingto the invention appear to be competitive at such bitrates, providinghigh audio quality at moderate computational expense.

As the skilled person realizes, the fact that the upmix stage is furtheroperable in a pass-through mode enables the decoder to adaptively decodeaccording to conventional direct or joint coding and complex predictioncoding, as determined on the encoder side. Hence, in those cases wherethe decoder cannot positively increase the level of quality beyond thatof conventional direct L/R stereo coding or joint M/S stereo coding, itcan at least guarantee that the same level is maintained. Thus, adecoder according to this aspect of the invention may, from a functionalpoint of view, be regarded as a superset in relation to the backgroundart.

As an advantage over QMF-based prediction-coded stereo, perfectreconstruction of the signal is possible (apart from quantizationerrors, which can be made arbitrarily small).

Thus, the invention provides coding apparatus for transform-based stereocoding by complex prediction. Preferably, an apparatus according to theinvention is not limited to complex prediction stereo coding, but isoperable also in a direct L/R stereo coding or joint M/S stereo codingregime according to the background art, so that it is possible to selectthe most suitable coding method for a particular application or during aparticular time interval.

An oversampled (e.g., complex) representation of the signal, includingboth said first and said second spectral components, is used as a basisfor the complex prediction according to the invention, and hence,modules for computing such oversampled representation are arranged inthe encoder system and the decoder system according to the invention.The spectral components refer to first and second subspaces of amultidimensional space, which may be the set of time-dependent functionson an interval of given length (e.g., a predefined time frame length)sampled at a finite sampling frequency. It is well-known that functionsin this particular multidimensional space may be approximated by afinite weighted sum of base functions.

As the skilled person will appreciate, an encoder adapted to cooperatewith a decoder is equipped with equivalent modules for providing theoversampled representation on which the prediction coding is based, soas to enable faithful reproduction of the encoded signal. Suchequivalent modules may be identical or similar modules or modules havingidentical or similar transfer characteristics. In particular, themodules in the encoder and decoder, respectively, may be similar ordissimilar processing units executing respective computer programs thatperform equivalent sets of mathematical operations.

In some embodiments of the decoder system or of the encoder system, thefirst spectral components have real values expressed in the firstsubspace, and the second spectral components have imaginary valuesexpressed in the second subspace. The first and second spectralcomponents together form a complex spectral representation of thesignal. The first subspace may be the linear span of a first set of basefunctions, while the second subspace may be the linear span of a set ofsecond base functions, some of which are linearly independent of thefirst set of base functions.

In one embodiment, the module for computing the complex representationis a real-to-imaginary transform, i.e., a module for computing imaginaryparts of the spectrum of a discrete-time signal on the basis of a realspectral representation of the signal. The transform may be based onexact or approximate mathematical relations, such as formulas fromharmonic analysis or heuristic relations.

In some embodiments of the decoder system or of the encoder system, thefirst spectral components are obtainable by a time-to-frequency domaintransform, preferably a Fourier transform, of a discrete time-domainsignal, such as by a discrete cosine transform (DCT), a modifieddiscrete cosine transform (MDCT), a discrete sine transform (DST), amodified discrete sine transform (MDST), a fast Fourier transform (FFT),a prime-factor-based Fourier algorithm or the like. In the first fourcases, the second spectral components may then be obtainable by DST,MDST, DCT and MDCT, respectively. As is well known, the linear span ofcosines that are periodic on the unit interval forms a subspace that isnot entirely contained in the linear span of sines periodic on the sameinterval. Preferably, the first spectral components are obtainable byMDCT and the second spectral components are obtainable by MDST.

In one embodiment, the decoder system includes at least one temporalnoise shaping module (TNS module, or TNS filter), which is arrangedupstream of the upmix stage. Generally speaking, the use of TNSincreases the perceived audio quality for signals with transient-likecomponents, and this also applies to embodiments of the inventivedecoder system featuring TNS. In conventional L/R and M/S stereo coding,the TNS filter may be applied as a last processing step in the frequencydomain, directly before the inverse transform. In case ofcomplex-prediction stereo coding, however, it is often advantageous toapply the TNS filter on the downmix and residual signals, that is,before the upmix matrix. Put differently, the TNS is applied to linearcombinations of the left and right channels, which has severaladvantages. Firstly, it may turn out in a given situation that TNS isonly beneficial for, say, the downmix signal. Then, TNS filtering can besuppressed or omitted for the residual signal and, what may mean moreeconomic use of the available bandwith, TNS filter coefficients needonly be transmitted for the downmix signal. Secondly, the computation ofthe oversampled representation of the downmix signal (e.g., MDST databeing derived from the MDCT data so as to form a complexfrequency-domain representation), which is needed in complex predictioncoding, may require that at time-domain representation of the downmixsignal be computable. This in turn means that the downmix signal ispreferably available as a time sequence of MDCT spectra obtained in auniform manner. If the TNS filter were applied in the decoder after theupmix matrix, which converts a downmix/residual representation into aleft/right representation, only a sequence of TNS residual MDCT spectraof the downmix signal would be available. This would make efficientcalculation of the corresponding MDST spectra very challenging,especially if left and right channels were using TNS filters withdifferent characteristics.

It is emphasized that the availability of a time sequence of MDCTspectra is not an absolute criterion in order to obtain an MDSTrepresentation fit to serve as a basis for complex prediction coding. Inaddition to experimental evidence, this fact may be explained by the TNSbeing generally applied only to higher frequencies, such as above a fewkilohertz, so that the residual signal filtered by TNS approximatelycorresponds to the non-filtered residual signal for lower frequencies.Thus, the invention may be embodied as a decoder for complex-predictionstereo coding, in which the TNS filters have a different placement thanupstream of the upmix stage, as indicated below.

In one embodiment, the decoder system includes at least one further TNSmodule located downstream of the upmix stage. By means of a selectorarrangement, either the TNS module(s) upstream of the upmix stage or theTNS module(s) downstream of the upmix stage. Under certaincircumstances, the computation of the complex frequency-domainrepresentation does not require that a time-domain representation of thedownmix signal be computable. Moreover, as set forth above, the decodermay be selectively operable in a direct or joint coding mode, notapplying complex prediction coding, and then it may be more suitable toapply the conventional localization of the TNS modules, that is, as oneof the last processing steps in the frequency domain.

In one embodiment, the decoder system is adapted to economize processingresources, and possibly energy, by deactivating the module for computinga second frequency-domain representation of the downmix signal when thelatter is not necessary. It is supposed that the downmix signal ispartitioned into successive time blocks, each of which is associatedwith a value of the complex prediction coefficient. This value may bedetermined by a decision taken for each time block by an encodercooperating with the decoder. Furthermore, in this embodiment, themodule for computing a second frequency-domain representation of thedownmix signal is adapted to deactivate itself if, for a given timeblock, the absolute value of the imaginary part of the complexprediction coefficient is zero or is smaller than a predeterminedtolerance. Deactivation of the module may imply that no secondfrequency-domain representation of the downmix signal is computed forthis time block. If deactivation did not take place, the secondfrequency-domain representation (e.g., a set of MDST coefficients) wouldbe multiplied by zero or by a number of substantially the same order ofmagnitude as the machine epsilon (round-off unit) of the decoder or someother suitable threshold value.

In a further development of the preceding embodiment, economization ofprocessing resources is achieved on a sublevel of the time block intowhich the downmix signal is partitioned. For instance, such a sublevelwithin a time block may be a frequency band, wherein the encoderdetermines a value of the complex prediction coefficient for eachfrequency band within a time block. Similarly, the module for producinga second frequency-domain representation is adapted to suppress itsoperation for a frequency band in a time block for which the complexprediction coefficient is zero or has magnitude less than a tolerance.

In one embodiment, the first spectral components are transformcoefficients arranged in one or more time blocks of transformcoefficients, each block generated by application of a transform to atime segment of a time-domain signal. Further, the module for computinga second frequency-domain representation of the downmix signal isadapted to:

-   -   derive one or more first intermediate components from at least        some of the first spectral components;    -   form a combination of said one or more first spectral components        according to at least a portion of one or more impulse responses        to obtain one or more second intermediate components; and    -   derive said one or more second spectral components from said one        or more second intermediate components.        This procedure achieves a computation of the second        frequency-domain representation directly from the first        frequency-domain representation, as described in greater detail        in U.S. Pat. No. 6,980,933 B2, notably columns 8-28 and in        particular equation 41 therein. As the skilled person realizes,        the computation is not performed via the time domain, as opposed        to, e.g., inverse transformation followed by a different        transformation.

For an exemplary implementation of complex-prediction stereo codingaccording to the invention, it has been estimated that the computationalcomplexity increases only slightly (significantly less than the increasecaused by complex-prediction stereo coding in the QMF domain) comparedto traditional L/R or M/S stereo. An embodiment of this type includingexact computation of the second spectral components introduces a delaythat is typically only a few per cent longer than that introduced by aQMF-based implementation (assuming the time block length to be 1024samples and comparing with the delay of the hybrid QMFanalysis/synthesis filter bank, which is 961 samples).

Suitably, in at least some of the previous embodiment, the impulseresponses are adapted to the transform by which the firstfrequency-domain representation is obtainable, and more precisely inaccordance with the frequency response characteristics thereof.

In some embodiments, the first frequency-domain representation of thedownmix signal is obtained by a transform which is being applied inconnection with one or more analysis window functions (or cut-offfunctions, e.g., rectangular window, sine window, Kaiser-Bessel-derivedwindow, etc.), one aim of which is to achieve a temporal segmentationwithout introducing a harmful amount of noise or changing the spectrumin an undesirable manner. Possibly, such window functions are partiallyoverlapping. Then, preferably, the frequency response characteristics ofthe transform are dependent on characteristics of said one or moreanalysis window functions.

Still referring to the embodiments featuring computation of the secondfrequency-domain representation within the frequency domain, it ispossible to decrease the computational load involved by using anapproximate second frequency-domain representation. Such approximationmay be achieved by not requiring complete information on which to basethe computation. By the teachings of U.S. Pat. No. 6,980,933 B2, forinstance, first frequency-domain data from three time blocks arerequired for exact calculation of the second frequency-domainrepresentation of the downmix signal in one block, namely a blockcontemporaneous with the output block, a preceding block and asubsequent block. For the purpose of complex prediction coding accordingto the present invention, suitable approximations may be obtained byomitting—or replacing by zero—data emanating from the subsequent block(whereby operation of the module may become causal, that is, does notcontribute a delay) and/or from the preceding block, so that thecomputation of the second frequency-domain representation is based ondata from one or two time blocks only. It is noted that even though theomission of input data may imply a rescaling of the secondfrequency-domain representation—in the sense that, e.g., it no longerrepresents equal power—it can yet be used as a basis for complexprediction coding as long as it is computed in an equivalent manner atboth the encoder and decoder ends, as noted above. Indeed, a possiblerescaling of this kind will be compensated by a corresponding change ofthe prediction coefficient value.

Yet another approximate method for computing a spectral componentforming part of the second frequency-domain representation of thedownmix signal may include combination of at least two components fromthe first frequency-domain representation.

The latter components may be adjacent with respect to time and/orfrequency. As alternative, they may be combined by finite impulseresponse (FIR) filtering, with relatively few taps. For example, in asystem applying a time block size of 1024, such FIR filters may include2, 3, 4 etc. taps. Descriptions of approximate computation methods ofthis nature may be found, e.g., in US 2005/0197831 A1. If a windowfunction giving relatively smaller weights to the neighborhood of eachtime block boundary is used, e.g., a non-rectangular function, it may beexpedient to base the second spectral components in a time block only oncombinations of first spectral components in the same time block,implying that not the same amount of information is available for theoutermost components. The approximation error possibly introduced bysuch practice is to some extent suppressed or concealed by the shape ofthe window function.

In one embodiment of a decoder, which is designed to output atime-domain stereo signal, there is included a possibility of switchingbetween direct or joint stereo coding and complex prediction coding.This is achieved by the provision of:

-   -   a switch that is selectively operable either as a pass-through        stage (not modifying the signals) or as a sum-and-difference        transform;    -   an inverse transform stage for performing a frequency-to-time        transform; and    -   a selector arrangement for feeding the inverse transform stage        with either a directly (or jointly) coded signal or with a        signal coded by complex prediction.        As the skilled person realizes, such flexibility on the part of        the decoder gives the encoder latitude to choose between        conventional direct or joint coding and complex prediction        coding. Hence, in cases where the level of quality of        conventional direct L/R stereo coding or joint M/S stereo coding        cannot be surpassed, this embodiment can at least guarantee that        the same level is maintained. Thus, the decoder according to        this embodiment may be regarded as a superset with respect to        the related art.

Another group of embodiments of the decoder system effect computation ofthe second spectral components in the second frequency-domainrepresentation via the time domain. More precisely, an inverse of thetransform by which the first spectral components were obtained (or areobtainable) is applied and is followed by a different transform havingas output the second spectral components. In particular, an inverse MDCTmay be followed by a MDST. In order to reduce the number of transformsand inverse transforms, the output of the inverse MDCT may, in such anembodiment, be fed to both the MDST and to the output terminals(possibly preceded by further processing steps) of the decoding system.

For an exemplary implementation of complex-prediction stereo codingaccording to the invention, it has been estimated that the computationalcomplexity increases only slightly (still significantly less than theincrease caused by complex-prediction stereo coding in the QMF domain)compared to traditional L/R or M/S stereo.

As a further development of the embodiment referred to in the precedingparagraph, the upmix stage may comprise a further inverse transformstage for processing the side signal. Then, the sum-and-difference stageis supplied with a time-domain representation of the side signal,generated by said further inverse transform stage, and a time-domainrepresentation of the downmix signal, generated by the inverse transformstage already referred to. It is recalled that, advantageously from thepoint of view of computational complexity, the latter signal is suppliedto both the sum-and-difference stage and said different transform stagereferred to above.

In one embodiment, a decoder designed to output a time-domain stereosignal includes a possibility of switching between direct L/R stereocoding or joint M/S stereo coding and complex prediction stereo coding.This is achieved by the provision of:

-   -   a switch operable either as a pass-through stage or as a        sum-and-difference stage;    -   a further inverse transform stage for computing a time-domain        representation of the side signal;    -   a selector arrangement for connecting the inverse transform        stages to either a further sum-and-difference stage connected to        a point upstream of the upmix stage and downstream of the switch        (preferably when the switch has been actuated to function as a        pass filter, as may be the case in decoding a stereo signal        generated by complex prediction coding) or a combination of a        downmix signal from the switch and a side signal from the        weighted summer (preferably when the switch has been actuated to        function as a sum-and-difference stage, as may be the case in        decoding a directly coded stereo signal).        As the skilled person realizes, this gives the encoder latitude        to choose between conventional direct or joint coding and        complex prediction coding, which means that a level of quality        at least equivalent to that of direct or joint stereo coding can        be guaranteed.

In one embodiment, of the encoder system according to the second aspectof the invention may comprise an estimator for estimating the complexprediction coefficient with the aim of reducing or minimizing the signalpower or average signal power of the residual signal. The minimizationmay take place over a time interval, preferably a time segment or timeblock or time frame of the signal to be encoded. The square of theamplitude may be taken as a measure of the momentary signal power, andan integral over a time interval of the squared amplitude (waveform) maybe taken as a measure of the average signal power in that interval.Suitably, the complex prediction coefficient is determined on atime-block and frequency-band basis, that is, its value is set in suchmanner that it reduces the average power (i.e., total energy) of theresidual signal in that time block and frequency band. In particular,modules for estimating parametric stereo coding parameters such as IID,ICC and IPD or similar ones, may provide output on which the complexprediction coefficient can be computed according to mathematicalrelations known to the skilled person.

In one embodiment, the coding stage of the encoder system is operable,further, to function as pass-through stage so as to enable direct stereocoding. By selecting direct stereo coding in situations where this isexpected to provide a higher quality, the encoder system can guaranteethat the coded stereo signal has at least the same quality as in directcoding. Similarly, in situations where the greater computational effortincurred by complex prediction coding is not motivated by a significantquality increase, an option of economizing computational resources isthus readily available to the encoder system. The decision betweenjoint, direct, real-prediction and complex-prediction coding in thecoder is generally based on a rate/distortion optimization rationale.

In one embodiment, the encoder system may comprise a module forcomputing a second frequency-domain representation directly (that is,without applying an inverse transform into the time domain and withoutusing the time-domain data of the signal) based on the first spectralcomponents. In relation to the corresponding embodiments of the decodersystem described above, this module may have an analogous structure,namely comprise the analogous processing operations but in a differentorder, so that the encoder is adapted to output data suitable as inputon the decoder side. For the purposes of illustrating this embodiment,it is assumed that the stereo signal to be encoded comprises mid andside channels, or has been transformed into this structure, and thecoding stage is adapted to receive a first frequency-domainrepresentation. The coding stage comprises a module for computing asecond frequency-domain representation of the mid channel. (The firstand second frequency-domain representations referred to here are asdefined above; in particular the first frequency-domain representationsmay MDCT representations and the second frequency-domain representationmay be an MDST representation.) The coding stage further comprises aweighted summer for computing a residual signal as a linear combinationformed from the side signal and the two frequency-domain representationsof the mid signal weighted by the real and imaginary parts,respectively, of the complex prediction coefficient. The mid signal, orsuitably the first frequency-domain representation thereof, may be useddirectly as a downmix signal. In this embodiment, further, the estimatordetermines the value of the complex prediction coefficient with the aimof minimizing the power or average power of the residual signal. Thefinal operation (optimization) may be effected either by feedbackcontrol, wherein the estimator may receive the residual signal obtainedby current prediction coefficient values to be adjusted further ifneeded, or, in a feed-forward manner, by computations effected directlyon the left/right channels of an original stereo signal or the mid/sidechannels. The feed-forward method is preferred, by which the complexprediction coefficient is determined directly (particularly, in anon-iterative or non-feedback manner) based on the first and secondfrequency-domain representations of the mid signal and the firstfrequency-domain representation of the side signal. It is noted that thedetermination of the complex prediction coefficient may be followed by adecision whether to apply direct, joint, real-prediction orcomplex-prediction coding, wherein the resulting quality (preferably theperceptual quality, taking into account, e.g., signal-to-mask effects)of each available option is considered; thus, the statements above arenot to be construed to the effect that no feedback mechanism exists inthe encoder.

In one embodiment, the encoder system comprises modules for computing asecond frequency-domain representation of the mid (or downmix) signalvia the time domain. It is understood that implementation detailsrelating to this embodiment, at least as far as the computation of thesecond frequency-domain representation is concerned, are similar or canbe worked out analogously to corresponding decoder embodiments. In thisembodiment, the coding stage comprises:

-   -   a sum-and-difference stage for converting the stereo signal into        a form comprising mid and side channels;    -   a transform stage for providing a frequency-domain        representation of the side channel and a complex-valued (and        hence oversampled) frequency-domain representation of the mid        channel; and    -   a weighted summer for computing a residual signal, wherein the        complex prediction coefficient is used as a weight.        Here, the estimator may receive the residual signal and        determine, possibly in a feedback control fashion, the complex        prediction coefficient so as to reduce or minimize the power or        average of the residual signal. Preferably, however, the        estimator receives the stereo signal to be encoded and        determines the prediction coefficient on the basis of this. It        is advantageous from the point of view of computational economy        to use a critically sampled frequency-domain representation of        the side channel, as the latter will not be subjected to        multiplication by a complex number in this embodiment. Suitably,        the transform stage may comprise an MDCT stage and an MDST stage        arranged in parallel, both having the time-domain representation        of the mid channel as input. Thus, an oversampled        frequency-domain representation of the mid channel and a        critically sampled frequency-domain representation of the side        channel are produced.

It is noted that the methods and apparatus disclosed in this section maybe applied, after appropriate modifications within the skilled person'sabilities including routine experimentation, to coding of signals havingmore than two channels. The modifications into such multi-channeloperability may proceed, e.g., along the lines of sections 4 and 5 inthe paper by J. Herre et al. cited above.

Features from two or more embodiments outlined above can be combined,unless they are clearly complementary, in further embodiments. The factthat two features are recited in different claim does not preclude thatthey can be combined to advantage. Likewise, further embodiments canalso be provided the omission of certain features that are not necessaryor not essential for the desired purpose. As one example, the decodingsystem according to the invention may be embodied without adequantization stage in cases where the coded signal to be processed isnot quantized or is already available in a form suitable for processingby the upmix stage.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be further illustrated by the embodimentsdescribed in the next section, reference being made to the accompanyingdrawings, on which:

FIG. 1A is a generalized block diagram showing a QMF-based decoderaccording to background art;

FIG. 1B is a generalized block diagram showing a QMF-based decoderaccording to background art;

FIG. 2 is a generalized block diagram of an MDCT-based stereo decodersystem with complex prediction, according to an embodiment of thepresent invention, in which the complex representation of a channel ofthe signal to be decoded is computed in the frequency domain;

FIG. 3 is a generalized block diagram of an MDCT-based stereo decodersystem with complex prediction, according to an embodiment of thepresent invention, in which the complex representation of a channel ofthe signal to be decoded is computed in the time domain;

FIG. 4 shows an alternative embodiment of the decoder system of FIG. 2,in which the location of the active TNS stage is selectable;

FIG. 5 comprises generalized block diagrams showing MDCT-based stereoencoder systems with complex prediction, according to embodiments ofanother aspect of the present invention;

FIG. 6 is a generalized block diagram of an MDCT-based stereo encoderwith complex prediction, according to an embodiment of the invention, inwhich a complex representation of a channel of the signal to be encodedis computed on the basis of the time-domain representation thereof;

FIG. 7 shows an alternative embodiment of the encoder system of FIG. 6,which is operable also in a direct L/R coding mode;

FIG. 8 is a generalized block diagram of an MDCT-based stereo encodersystem with complex prediction, according to an embodiment of theinvention, in which a complex representation of a channel of the signalto be encoded is computed on the basis of a first frequency-domainrepresentation thereof, which decoder system is operable also in adirect L/R coding mode;

FIG. 9 shows an alternative embodiment of the encoder system of FIG. 7,which further includes a TNS stage arranged downstream of the codingstage;

FIG. 10 shows alternative embodiments of the portion labeled A in FIGS.2 and 8;

FIG. 11 is shows an alternative embodiment of the encoder system of FIG.8, which further includes two frequency-domain modifying devicesrespectively arranged downstream and upstream of the coding stage;

FIG. 12 is a graphical presentation of listening test results at 96 kb/sfrom six subjects showing different complexity—quality trade-off optionsfor the computation or approximation of the MDST spectrum, wherein datapoints labeled “+” refer to hidden reference, “×” refer to 3.5 kHzband-limited anchor, “*” refer to USAC traditional stereo (M/S or L/R),“□” refer to MDCT-domain unified stereo coding by complex predictionwith imaginary part of prediction coefficient disabled (i.e.,real-valued prediction, requiring no MDST), “▪” refer to MDCT-domainunified stereo coding by complex prediction using a current MDCT frameto compute an approximation of the MDST, “◯” refer to MDCT-domainunified stereo coding by complex prediction using current and previousMDCT frames to compute an approximation of the MDST and “●” refer toMDCT-domain unified stereo coding by complex prediction using current,previous and next MDCT frames to compute the MDST;

FIG. 13 presents the data of FIG. 12, however as differential scoresrelative to MDCT-domain unified stereo coding by complex predictionusing a current MDCT frame to compute an approximation of the MDST;

FIG. 14A is a generalized block diagram showing an embodiment of adecoder system according to an embodiment of the invention;

FIG. 14B is a generalized block diagram showing an embodiment of adecoder system according to an embodiment of the invention;

FIG. 14C is a generalized block diagram showing an embodiment of adecoder system according to an embodiment of the invention;

FIG. 15 is a flowchart showing a decoding method according to anembodiment of the invention; and

FIG. 16 is a flowchart showing an encoding method according to anembodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS I. Decoder Systems

FIG. 2 shows, in the form of a generalized block diagram, a decodingsystem for decoding a bit stream comprising at least one value of acomplex prediction coefficient α=α_(R)+iα_(t) and an MDCT representationof a stereo signal having downmix M and residual D channels. The realand imaginary parts α_(R), α_(l) of the prediction coefficient may havebeen quantized and/or coded jointly. Preferably however, the real andimaginary parts are quantized independently and uniformly, typicallywith a step size of 0.1 (dimensionless number). The frequency-bandresolution used for the complex prediction coefficient is notnecessarily the same as the resolution for scale factors bands (sfb;i.e., a group of MDCT lines that are using the same MDCT quantizationstep size and quantization range) according to the MPEG standard. Inparticular, the frequency-band resolution for the prediction coefficientmay be one that is psycho-acoustically justified, such as the Barkscale. A demultiplexer 201 is adapted to extract these MDCTrepresentations and the prediction coefficient (part of Controlinformation as indicated in the figure) from the bit stream that issupplied to it. Indeed, more control information than merely the complexprediction coefficient may be encoded in the bit stream, e.g.,instructions whether the bit stream is to be decoded in prediction ornon-prediction mode, TNS information, etc. TNS information may includevalues of the TNS parameters to be applied by the TNS (synthesis)filters of the decoder system. If identical sets of TNS parameters areto be used for several TNS filters, such as for both channels, it iseconomical receive this information in the form of a bit indicating suchidentity of the parameter sets rather than receiving the two sets ofparameters independently. Information may also be included whether toapply TNS before or after the upmix stage, as appropriate based on,e.g., a psycho-acoustic evaluation of the two available options.Moreover, then control information may indicate individually limitedbandwidths for the downmix and residual signals. For each channel,frequency bands above a bandwidth limit will not be decoded but will beset to zero. In certain cases, the highest frequency bands have so smallenergy content that they are already quantized down to zero. Normalpractice (cf. the parameter max_sfb in the MPEG standard) has been touse the same bandwidth limitation for both the downmix and residualsignals. However, the residual signal, to a greater extent than thedownmix signal, has its energy content localized to lower frequencybands. Therefore, by placing a dedicated upper bandwidth limit on theresidual signal, a bit-rate reduction is possible at no significant lossof quality. For instance, this may be governed by two independentmax_sfb parameters encoded in the bit stream, one for the downmix signaland one for the residual signal.

In this embodiment, the MDCT representation of the stereo signal issegmented into successive time frames (or time blocks) comprising afixed number of data points (e.g., 1024 points), one of several fixednumbers of data points (e.g., 128 or 1024 points) or a variable numberof points. As is known to those skilled in the art, the MDCT iscritically sampled. The output of the decoding system, indicated in theright part of the drawing, is a time-domain stereo signal having left Land right R channels. Dequantization modules 202 are adapted to handlethe bit stream input to the decoding system or, where appropriate, twobit streams obtained after demultiplexing of an original bit stream andcorresponding to each of the downmix and residual channels. Thedequantized channel signals are provided to a switching assembly 203operable either in a pass-through mode or a sum-and-difference modecorresponding to the respective transformation matrices

$\begin{bmatrix}1 & 0 \\0 & 1\end{bmatrix}\mspace{14mu} {and}\mspace{14mu} {{\frac{1}{2}\begin{bmatrix}1 & 0 \\0 & {- 1}\end{bmatrix}}.}$

As will be further discussed in the next paragraph, the decoder systemincludes a second switching assembly 205. Both switching assemblies 203,205, like most other switches and switching assemblies in thisembodiment and the embodiments to be described, are operable in afrequency-selective fashion. This enables decoding of a large variety ofdecoding modes, e.g., decoding frequency-dependent L/R or M/S decoding,as known in the related art. Hence, the decoder according to theinvention can be regarded as a superset in relation to the related art.

Assuming for now that the switching assembly 203 is in the pass-throughmode, the dequantized channel signals are passed, in this embodiment,through respective TNS filters 204. The TNS filters 204 are notessential to the operation of the decoding system and may be replaced bypass-through elements. After this, the signal is supplied to the secondswitching assembly 205 having the same functionality as the switchingassembly 203 located upstream. With inputs signals as previouslydescribed and with the second switching assembly 205 set in itspass-through mode is, the output of the former is the downmix channelsignal and the residual channel signal. The downmix signal, stillrepresented by its time-successive MDCT spectra, is supplied to areal-to-imaginary transform 206 adapted to compute, based thereon, MDSTspectra of the downmix signal. In this embodiment, one MDST frame isbased on three MDCT frames, one previous frame, one current (orcontemporaneous) frame and one subsequent frame. It is indicatedsymbolically (Z⁻¹, Z) that the input side of the real-to-imaginarytransform 206 comprises delay components.

The MDST representation of the downmix signal obtained from thereal-to-imaginary transform 206 is weighted by the imaginary part α_(t)of the prediction coefficient and is added to the MDCT representation ofthe downmix signal weighted by the real part α_(R) of the predictioncoefficient and the MDCT representation of the residual signal. The twoadditions and multiplications are performed by multipliers and adders210, 211, together forming (functionally) a weighted adder, which aresupplied with the value of the complex prediction coefficient α encodedin the bit stream initially received by the decoder system. The complexprediction coefficient may be determined once for every time frame. Itmay also be determined more often, such as once for every frequency bandwithin a frame, the frequency bands being a psycho-acousticallymotivated partition. It may also be determined less frequently, as willbe discussed below in connection with encoding systems according to theinvention. The real-to-imaginary transform 206 is synchronized with theweighted adder in such manner that a current MDST frame of the downmixchannel signal is combined with one contemporaneous MDCT frames of eachof the downmix channel signal and the residual channel signal. The sumof these three signals is a side signal S=Re(αM)+D. In this expression,M includes both the MDCT and MDST representations of the downmix signal,namely M=M_(MDCT)−lM_(MDST), whereas D=D_(MDCT) is real-valued. Thus, astereo signal having a downmix channel and a side channel is obtained,from which a sum-and-difference transform 207 restores the left andright channels as follows:

$\quad \left\{ \begin{matrix}{L = {M + S}} \\{R = {M - S}}\end{matrix} \right.$

These signals are represented in the MDCT domain. The last step of thedecoding system is to apply an inverse MDCT 209 to each of the channels,whereby a time-domain representation of the left/right stereo signal isobtained.

A possible implementation of the real-to-imaginary transform 206 isfurther described in applicant's patent U.S. Pat. No. 6,980,933 B2, asnoted above. By formula 41 therein, the transform can be expressed as afinite impulse-response filter, e.g., for even points,

${S\left( {2v} \right)} = {{\frac{2}{N}{\sum\limits_{p = 0}^{N - 1}{\left\lbrack {{\left( {- 1} \right)^{p + 1}{X_{I}(p)}} + {X_{III}(p)}} \right\rbrack {h_{I,{III}}\left( {{2v} - p} \right)}}}} + {\frac{4}{N}{\sum\limits_{l = 0}^{N - 1}{{X_{II}\left( {{2l} + 1} \right)}{{h_{II}\left( {{2v} - {2l} - 1} \right)}.}}}}}$

where S(2v) is the 2v^(th) MDST data point, X_(I),X_(IIα),X_(m) are theMDCT data from each of the frames and N is the frame length. Further,h_(l,m)+h_(u) are impulse responses depending on the window functionapplied, and are therefore determined for each choice of windowfunction, such as rectangular, sinusoidal and Kaiser-Bessel-derived, andfor each frame length. The complexity of this computation may be reducedby omitting those impulse responses which have a relatively smallerenergy content and contribute relatively less to the MDST data. As analternative or extension to this simplification, the impulse responsesthemselves may be shortened, e.g., from the full frame length N tosmaller number of points. As an example, the impulse response length maybe decreased from 1024 points (taps) to 10 points. The most extremetruncation that can still be considered meaningful is

S(v)+X _(u)(v+1)−X _(n)(v−1),

Other straightforward approaches can be found in US 2005/0197831 A1.

It is further possible to reduce the amount of input data on which thecomputation is based. To illustrate, the real-to-imaginary transform 206and its upstream connections, which are indicated as a portion denotedby “A” on the drawing, be replaced by a simplified variant, two of whichA′ and A″ are shown in FIG. 10. The variant A′ provides an approximateimaginary representation of the signal. Here, the MDST computation onlytakes the current and the previous frame into account. With reference tothe formula above in this paragraph, this may be effected by settingX_(m)(p)=0 for p=0, . . . , N−1 (index III denotes the subsequent timeframe). Because the variant A′ does not require MDCT spectrum of thesubsequent frame as input, the MDST calculation does not incur any timedelay. Clearly, this approximation somewhat reduces the accuracy of theMDST signal obtained, but may also imply that the energy of this signalbecomes reduced; the latter fact can be fully compensated by an increasein α_(l) as a result of the nature of prediction coding.

Also shown in FIG. 10 is the variant A″, which uses as input only theMDCT data for the current time frame. Variant A″ arguably produces aless accurate MDST representation than variant A′. On the other hand, itoperates at zero delay, just like variant A′, and has lowercomputational complexity. As already mentioned, the waveform codingproperties are not affected as long as the same approximation is used inthe encoder system and the decoder system.

It is to be noted, irrespective of whether variant A, A′ or A″ or anyfurther development thereof is used, that only those portions of theMDST spectrum need to be computed for which the imaginary part of thecomplex prediction coefficient is non-zero, α_(l)≠0 . In practicalcircumstances this will be taken to mean that the absolute value |α_(l)|of the imaginary part of the coefficient is greater than a predeterminedthreshold value, which may be related to the unit round-off of thehardware used. In case the imaginary part of the coefficient is zero forall frequency bands within a time frame, there is no need to compute anyMDST data for that frame. Thus, suitably, the real-to-imaginarytransform 206 is adapted to respond to occurrences of very small |α_(l)|values by not generating MDST output, whereby computing resources can beeconomized. In embodiments where more frames than the current one areused to produce one frame of MDST data, however, any units upstream ofthe transform 206 should suitably continue operating even though no MDSTspectrum is needed—in particular, the second switching assembly 205should keep forwarding MDCT spectra—so that sufficient input data arealready available to the real-to-imaginary transform 206 already whenthe next time frame associated with a non-zero prediction coefficientoccurs; this may of course be the next time block.

Returning to FIG. 2, the function of the decoding system has beendescribed under the assumption of both switching assemblies 203, 205being set in their respective pass-through modes. As will be discussednow, the decoder system can as well decode signals that are notprediction coded. For this use, the second switching assembly 205 willbe set in its sum-and-difference mode and suitably, as indicated on thedrawing, a selector arrangement 208 will be set in its lower position,thereby ensuring that signals are fed directly to the inverse transform209 from a source point between the TNS filters 204 and the secondswitching assembly 205. To ensure correct decoding, the signal suitablyhas L/R form at the source point. Therefore, to ensure that thereal-to-imaginary transform is supplied with the correct mid (i.e.,downmix) signal at all times (rather than, say, intermittently by a leftsignal), the second switching assembly 205 is preferably set in itssum-and-difference mode during decoding of a non-prediction-coded stereosignal. As noted above, prediction coding may be replaced byconventional direct or joint coding for certain frames based on, e.g., adata rate-to-audio quality decision. The outcome of such decision may becommunicated from the encoder to the decoder in various ways, e.g., bythe value of a dedicated indicator bit in each frame, or by the absenceor presence of a value of the prediction coefficient. Having establishedthese facts, the role of the first switching assembly 203 can be easilyrealized. Indeed, in non-prediction coding mode, the decoder system canprocess both signals according to direct (L/R) stereo coding or joint(M/S) coding, and by operating the first switching assembly 203 eitherin pass-through or sum-and-difference mode, it is possible to ensurethat the source point is always provided with a directly coded signal.Clearly, the switching assembly 203 when functioning assum-and-difference stage will convert an input signal in M/S form intoan output signal (supplied to the optional TNS filters 204) in L/R form.

The decoder system receives a signal whether a particular time frame isto be decoded by the decoder system in prediction-coding ornon-prediction-coding mode. Non-prediction mode may be signaled by thevalue of a dedicated indicator bit in each frame or by the absence (orthe value zero) of the prediction coefficient. Prediction mode may becommunicated analogously. A particularly advantageous implementation,which enables fallback without any overhead, makes use of a reservedfourth value of the two-bit field ms_mask_present (see MPEG-2 AAC,document ISO/IEC 13818-7), which is transmitted per time frame anddefined as follows:

TABLE 1 Definition of ms_mask_present in USAC Value Meaning 00 L/Rcoding for all frequency bands 01 one signaling bit per band is used toindicate L/R or M/S 10 M/S coding for all frequency bands 11 reservedBy redefining the value 11 to mean “complex prediction coding”, thedecoder can be operated in all legacy modes, particularly M/S and L/Rcoding, without any bit-rate penalty and is yet able to receive a signalindicating complex prediction coding mode for the relevant frames.

FIG. 4 shows a decoder system of the same general structure as the oneshown in FIG. 2 but including, however, at least two differentstructures. Firstly, the system of FIG. 4 includes switches 404, 411enabling the application of some processing step involvingfrequency-domain modification upstream and/or downstream of the upmixstage. This is achieved, on the one hand, by a first set offrequency-domain modifiers 403 (drawn as TNS synthesis filters in thisfigure) provided together with the first switch 404 downstream ofdequantization modules 401 and a first switching assembly 402 butupstream of a second switching assembly 405 arranged immediatelyupstream of the upmix stage 406, 407, 408, 409. On the other hand, thedecoder system includes a second set of frequency-domain modifiers 410provided together with a second switch 411 downstream of the upmix stage406, 407, 408, 409 but upstream of an inverse transform stage 412.Advantageously, as indicated on the drawing, each frequency-domainmodifier is arranged in parallel with a pass-through line which isconnected upstream to the input side of the frequency-domain modifierand is connected downstream to the associated switch. By virtue of thisstructure, the frequency-domain modifier is supplied with the signaldata at all times, enabling processing in the frequency domain based onmore time frames than the current one only. The decision whether toapply the first 403 or second sets of frequency-domain modifiers 410 maybe taken by the encoder (and conveyed in the bit stream), or may bebased on whether prediction coding is applied, or may be based on someother criterion found suitable in practical circumstances. As anexample, if the frequency-domain modifier are TNS filters, then thefirst set 403 may be advantageous to use for some kinds of signals,while the second set 410 may be advantageous for other kinds of signals.If the outcome of this selection is encoded in the bit stream, then thedecoder system will activate the respective set of TNS filtersaccordingly.

To facilitate understanding of the decoder system shown in FIG. 4, it isexplicitly noted that decoding of a directly (L/R) coded signal takesplace when α=0 (implying that pseudo-L/R and L/R are identical and thatthe side and residual channels do not differ), the first switchingassembly 402 is in the pass mode, the second switching assembly is inthe sum-and-difference mode, thereby causing the signal to have M/S formbetween the second switching assembly 405 and a sum-and-difference stage409 of the upmix stage. Because the upmix stage will then effectively bea pass-through step, it is immaterial whether (using the respectiveswitches 404, 411) the first or second set frequency-domain modifiers isactivated.

FIG. 3 illustrates a decoder system according to an embodiment of theinvention which, in relation to those of FIGS. 2 and 4, represents adifferent approach to the provision of MDST data required for theupmixing. Like the decoder systems already described, the system of FIG.3 comprises dequantization modules 301, a first switching assembly 302operable in either a pass-through or sum-and-difference mode and TNS(synthesis) filters 303, which are all serially arranged from the inputend of the decoder system. Modules downstream of this point areselectively utilized by means of two second switches 305, 310, which arepreferably operated jointly so that both are either in their upperpositions or lower positions, as indicated in the figure. At the outputend of the decoder system, there are a sum-and-difference stage 312 and,immediately upstream thereof, two inverse MDCT modules 306, 311 fortransforming an MDCT-domain representation of each channel into atime-domain representation.

In complex prediction decoding, wherein the decoder system is suppliedwith a bit stream encoding a downmix/residual stereo signal and valuesof a complex prediction coefficient, the first switching assembly 302 isset in its pass-through mode and the second switches 305, 310 are set inthe upper position. Downstream of the TNS filters, the two channels ofthe (dequantized, TNS-filtered, MDCT) stereo signal are processed indifferent ways. The downmix channel is provided, on the one hand, to amultiplier and summer 308, which adds the MDCT representation of thedownmix channel weighted by the real part α_(R) of the predictioncoefficient to the MDCT representation of the residual channel, and, onthe other hand, to one 306 of the inverse MDCT transform modules. Thetime-domain representation of the downmix channel M, which is outputfrom the inverse MDCT transform module 306, is supplied both to thefinal sum-and-difference stage 312 and to an MDST transform module 307.This double use of the time-domain representation of the downmix channelis advantageous from the point of view of computational complexity. TheMDST representation of the downmix channel thus obtained is supplied toa further multiplier and summer 309, which after weighting by theimaginary part α_(l) of the prediction coefficient adds this signal tothe linear combination output from the summer 308; hence, the output ofthe summer 309 is a side channel signal, S=Re(αM)+D. Similarly to thedecoder system shown in FIG. 2, the multipliers and summers 308, 309 mayreadily be combined to form a weighted multi-signal summer with inputsthe MDCT and MDST representations of the downmix signal, the MDCTrepresentation of the residual signal and the value of the complexprediction coefficient. Downstream of this point in the presentembodiment, only a passage through the inverse MDCT transform module 311remains before the side channel signal is supplied to the finalsum-and-difference stage 312.

The necessary synchronicity in the decoder system may be achieved byapplying the same transform lengths and window shapes at both inverseMDCT transform modules 306, 311, as is already the practice infrequency-selective M/S and L/R coding. A one-frame delay is introducedby the combination of certain embodiments of the inverse MDCT module 306and embodiments of the MDST module 307. Therefore, five optional delayblocks 313 (or software instructions to this effect in a computerimplementation) are provided, so that the portion of the system locatedto the right of the dashed line can be delayed by one frame in relationto the left portion when necessary. Apparently, all intersectionsbetween the dashed line and connection lines are provided with delayblocks, with the exception of the connection line between the inverseMDCT module 306 and the MDST transform module 307, which is where thedelay arises that requires compensation.

The computation of MDST data for one time frame requires data from oneframe of the time-domain representation. However, the inverse MDCTtransform is based on one (current), two (preferably: previous andcurrent) or three (preferably: previous, current and subsequent)consecutive frames. By virtue of the well-known time-domain aliascancellation (TDAC) associated with the MDCT, the three-frame optionachieves complete overlap of the input frames and thus provides the best(and possibly perfect) accuracy, at least in frames containingtime-domain alias. Clearly, the three-frame inverse MDCT operates at aone-frame delay. By accepting to use an approximate time-domainrepresentation as input to the MDST transform, one may avoid this delayand thereby the need to compensate delays between different portions ofthe decoder system. In the two-frame option, the overlap/add enablingTDAC occurs in the earlier half of the frame, and alias may be presentonly in the later half. In the one-frame option, the absence of TDACimplies that alias may occur throughout the frame; however, an MDSTrepresentation achieved in this manner, and used as an intermediatesignal in complex prediction coding, may still provide a satisfactoryquality.

The decoding system illustrated in FIG. 3 may also be operated in twonon-prediction decoding modes. For decoding a directly L/R coded stereosignal, the second switches 305, 310 are set in the lower position andthe first switching assembly 302 is set in the pass-through mode. Thus,the signal has L/R form upstream of the sum-and-difference stage 304,which converts it into M/S form, upon which inverse MDCT transformationand a final sum-and-difference operation take place. For decoding astereo signal provided in jointly M/S coded form, the first switchingassembly 302 is instead set in its sum-and-difference mode, so that thesignal has L/R form between the first switching assembly 302 and thesum-and-difference stage 304, which is often more suitable from thepoint of view of TNS filtering than an M/S form would be. The processingdownstream of the sum-and-difference stage 304 is identical to that inthe case of direct L/R decoding.

FIG. 14 consists of three generalized block diagrams of decodersaccording to embodiments of the invention. In contrast to several otherblock diagrams accompanying this application, a connection line in FIG.14 may symbolize a multi-channel signal. In particular, such connectionline may be arranged to transmit a stereo signal comprising left/right,mid/side, downmix/residual, pseudo-left/pseudo-right channels and othercombinations.

FIG. 14A shows a decoder system for decoding a frequency-domainrepresentation (indicated, for the purpose of this figure, as an MDCTrepresentation) of an input signal. The decoder system is adapted tosupply as its output a time-domain representation of a stereo signal,which is generated on the basis of the input signal. To be able todecode an input signal coded by complex prediction stereo coding, thedecoder system is provided with an upmix stage 1410. However, it is alsocapable of handling an input signal encoded in other formats andpossibly, that alternates between several coding formats over time,e.g., a sequence of time frames coded by complex prediction coding maybe followed by a time portion coded by direct left/right coding. Thedecoder system's ability to handle different coding formats is achievedby the provision of a connection line (pass-through) arranged inparallel with said upmix stage 1410. By means of a switch 1411 it ispossible to select whether the output from the upmix stage 1410 (lowerswitch position in figure) or the non-processed signal available overthe connection line (upper switch position in figure) is to be suppliedto the decoder modules arranged further downstream. In this embodiment,an inverse MDCT module 1412 is arranged downstream of the switch, whichtransforms an MDCT representation of a signal into a time-domainrepresentation. As an example, the signal supplied to the upmix stage1410 may be a stereo signal in downmix/residual form. The upmix stage1410 then is adapted to derive a side signal and to perform asum-and-difference operation so that a left/right stereo signal (in theMDCT domain) is output.

FIG. 14B shows a decoder system similar to that of FIG. 14A. The presentsystem is adapted to receive a bit stream at its input signal. The bitstream is initially processed by a combined demultiplexer anddequantization module 1420, which provides, as a first output signal, anMDCT representation of a multi-channel stereo signal for furthertreatment, as determined by the position of a switch 1422 havinganalogous functionality as the switch 1411 of FIG. 14A. More precisely,the switch 1422 determines whether the first output from thedemultiplexer and dequantization is to be processed by an upmix stage1421 and an inverse MDCT module 1423 (lower position) or by the inverseMDCT module 1423 only (upper position). The combined demultiplexer anddequantization module 1420 outputs control information as well. In thepresent case, the control information associated with the stereo signalmay include data indicating whether the upper or lower position of theswitch 1422 is suitable for decoding the signal or, more abstractly,according to what coding format the stereo signal is to be decoded. Thecontrol information may also include parameters for adjusting theproperties of the upmix stage 1421, e.g., a value of the complexprediction coefficient α used in complex prediction coding as alreadydescribed above.

FIG. 14C shows a decoder system which, in addition to the entitiesanalogous to those in FIG. 14B, comprises first and secondfrequency-domain modifying devices 1431, 1435 respectively arrangedupstream and downstream of an upmix stage 1433. For the purposes of thisfigure, each frequency-domain modifying device is illustrated by a TNSfilter. However, by the term frequency-domain modifying device couldalso be understood other processes than TNS filtering that aresusceptible of being applied either before or after the upmix stage.Examples of frequency-domain modifications include prediction, noiseaddition, bandwidth extension, and non-linear processing. Psychoacousticconsiderations and similar reasons, which possibly include theproperties of the signal to be processed and/or the configuration orsettings of such a frequency-domain modifying device, sometimes indicatethat it is advantageous to apply said frequency-domain modificationupstream of the upmix stage 1433 rather than downstream. In other cases,it may be established by similar considerations that the downstreamposition of the frequency-domain modification is preferable to theupstream one. By means of switches 1432, 1436, the frequency-domainmodifying devices 1431, 1435 may be selectively activated so that,responsive to control information, the decoder system can select thedesired configuration. As an example, FIG. 14C shows an configuration inwhich the stereo signal from the combined demultiplexer anddequantization module 1430 is initially processed by the firstfrequency-domain modifying device 1431, is then supplied to the upmixstage 1433 and is finally forwarded directly an inverse MDCT module1437, without passing through the second frequency-domain modifyingdevice 1435. As explained in section Summary, this configuration ispreferred over the option of performing TNS after upmixing in complexprediction coding.

II. Encoder Systems

An encoder system according to the invention will now be described withreference to FIG. 5, which is a generalized block diagram of an encodersystem for encoding a left/right (L/R) stereo signal as an output bitstream by complex prediction coding. The encoder system receives atime-domain or frequency-domain representation of the signal andsupplies this to both a downmix stage and a prediction coefficientestimator. The real and imaginary parts of the prediction coefficientsare provided to the downmix stage in order to govern the conversion ofthe left and right channels into downmix and residual channels, whichare then supplied to a final multiplexer MUX. If the signal was notsupplied as a frequency-domain representation to the encoder, it istransformed into such representation in the downmix stage ormultiplexer.

One of the principles in prediction coding is to convert the left/rightsignal to mid/side form, that is,

$\quad \left\{ \begin{matrix}{M = \frac{L + R}{2}} \\{S = \frac{L - R}{2}}\end{matrix} \right.$

and then to make use of the remaining correlation between thesechannels, namely by setting

S=Re(αM)+D,

where α is the complex prediction coefficient to be determined and D isthe residual signal. It is possible to choose α in order that the energyof the residual signal D=S−Re(αM) is minimized. Energy minimization maybe effected with respect to the momentary power, a shorter- orlonger-term energy (power average), which in the case of a discretesignal amounts to optimization in the mean-square sense.

The real and imaginary parts α_(R), α_(l) of the prediction coefficientmay be quantized and/or coded jointly. Preferably however, the real andimaginary parts are quantized independently and uniformly, typicallywith a step size of 0.1 (dimensionless number). The frequency-bandresolution used for the complex prediction coefficient is notnecessarily the same as the resolution for scale factors bands (sfb;i.e., a group of MDCT lines that are using the same MDCT quantizationstep size and quantization range) according to the MPEG standard. Inparticular, the frequency-band resolution for the prediction coefficientmay be one that is psycho-acoustically justified, such as the Barkscale. It is noted that the frequency-band resolution may vary in casesthe transform length varies.

As noted already, the encoder system according to the invention may havea latitude whether to apply prediction stereo coding or not, the lattercase implying a fall-back to L/R or M/S coding. Such decision may betaken on a time-frame basis or finer, on a frequency-band basis within atime frame. As noted above, a negative outcome of the decision may becommunicated to the decoding entity in various ways, e.g., by the valueof a dedicated indicator bit in each frame, or by the absence (or zerovalue) of a value of the prediction coefficient. A positive decision maybe communicated analogously. A particularly advantageous implementation,which enables fallback without any overhead, makes use of a reservedfourth value of the two-bit field ms_mask_present (see MPEG-2 AAC,document ISO/IEC 131818-7), which is transmitted per time frame anddefined as follows:

TABLE 1 Definition of ms_mask_present in USAC Value Meaning 00 L/Rcoding for all frequency bands 01 one signaling bit per band is used toindicate L/R or M/S 10 M/S coding for all frequency bands 11 reservedBy redefining the value 11 to mean “complex prediction coding”, theencoder can be operated in all legacy modes, particularly M/S and L/Rcoding, without any bit-rate penalty and is yet able to signal complexprediction coding for those frames where it is advantageous.

The substantive decision may be based on a data rate-to-audio qualityrationale. As a quality measure, data obtained using a psychoacousticmodel included in the encoder (as is often the case of availableMDCT-based audio encoders) may be used. In particular, some embodimentsof the encoder provides a rate-distortion optimized selection of theprediction coefficient. Accordingly, in such embodiments, the imaginarypart—and possibly the real part too—of the prediction coefficient is setto zero if the increase in prediction gain does not economize enoughbits for the coding of the residual signal to justify spending the bitsrequired for coding the prediction coefficient.

Embodiments of the encoder may encode information relating to TNS in thebit stream. Such information may include values of the TNS parameters tobe applied by the TNS (synthesis) filters on the decoder side. Ifidentical sets of TNS parameters are to be used for both channels, it iseconomical to include a signaling bit indicating this identity of theparameter sets rather than to transmit the two sets of parametersindependently. Information may also be included whether to apply TNSbefore or after the upmix stage, as appropriate based on, e.g., apsychoacoustic evaluation of the two available options.

As yet another optional feature, which is potentially beneficial from acomplexity and bit-rate point of view, the encoder may be adapted to usean individually limited bandwidth for the encoding of the residualsignal. Frequency bands above this limit will not be transmitted to thedecoder but will be set to zero. In certain cases, the highest frequencybands have so small energy content that they are already quantized downto zero. Normal practice (cf. the parameter max_sfb in the MPEGstandard) has entailed using the same bandwidth limitation for both thedownmix and residual signals. Now, the inventors have found empiricallythat the residual signal, to a greater extent than the downmix signal,has its energy content localized to lower frequency bands. Therefore, byplacing a dedicated upper band-with limit on the residual signal, abit-rate reduction is possible at no significant loss of quality. Forinstance, this may be achieved by transmitting two independent max_sfbparameters, one for the downmix signal and one for the residual signal.

It is pointed out that although the issues of optimal determination ofthe prediction coefficient, quantization and coding thereof, fallback tothe M/S or L/R mode, TNS filtering and upper bandwidth limitation etc.were discussed with reference to the decoder system shown in FIG. 5, thesame facts are equally applicable to the embodiments that will bedisclosed in what follows with reference to the subsequent figures.

FIG. 6 shows another encoder system according to the invention adaptedto perform complex prediction stereo coding. The system receives asinput a time-domain representation of a stereo signal segmented intosuccessive, possibly overlapping, time frames and comprising left andright channels. A sum-and-difference stage 601 converts the signal intomid and side channels. The mid channel is supplied to both an MDCTmodule 602 and an MDST module 603, while the side channel is supplied toan MDCT module 604 only. A prediction coefficient estimator 605estimates for each time frame—and possibly for individual frequencybands within a frame—a value of the complex prediction coefficient α asexplained above. The value of the coefficient α is supplied as weight toweighted summers 606, 607, which form a residual signal D as a linearcombination of the MDCT and MDST representations of the mid signal andthe MDCT representation of the side signal. Preferably, the complexprediction coefficient is supplied to the weighted summers 606, 607represented by the same quantization scheme which will be used when itis encoded into the bit stream; this obviously provides more faithfulreconstruction, as both encoder and decoder applies the same value ofthe prediction coefficient. The residual signal, the mid signal (whichmay be more appropriately called downmix signal when it appears incombination with a residual signal) and the prediction coefficient aresupplied to a combined quantization and multiplexer stage 608, whichencodes these and possible additional information as an output bitstream.

FIG. 7 shows a variation to the encoder system of FIG. 6. As is clearfrom the similarity of symbols in the figure, it has as similarstructure but also the added functionality of operating in a direct L/Rcoding fallback mode. The encoder system is actuated between the complexprediction coding mode and the fallback mode by means of a switch 710provided immediately upstream of the combined quantization andmultiplexer stage 709. In its upper position, as shown in the figure,the switch 710 will cause the encoder to operate in the fallback mode.From points immediately downstream of the MDCT modules 702, 704, themid/side signal is supplied to a sum-and-difference stage 705, whichafter converting it into left/right form passes it on to the switch 710,which connects it to the combined quantization and multiplexer stage709.

FIG. 8 shows an encoder system according to the present invention. Incontrast to the encoder systems of FIGS. 6 and 7, this embodimentderives the MDST data required for the complex prediction codingdirectly from the MDCT data, that is, by a real-to-imaginary transformin the frequency domain. The real-to-imaginary transform applies any ofthe approaches discussed in connection with the decoder systems of FIGS.2 and 4. It is important to match the computation method of the decoderwith that of the encoder, so that faithful decoding can be achieved;preferably, identical real-to-imaginary transform methods are used onthe encoder side and the decoder side. As for the decoder embodiments,the portion A enclosed by a dashed line and comprising thereal-to-imaginary transform 804 can be replaced by approximate variantsor using fewer input time frames as input. Likewise, the coding may besimplified using any one of the other approximation approaches describedabove.

On a higher level, the encoder system of FIG. 8 has a structurediffering from that which would probably follow by a straightforwardaction of replacing the MDST module in FIG. 7 by a (suitably connected)real-to-imaginary module. The present architecture is clean and achievesthe functionality of switching between prediction coding and direct L/Rcoding in a robust and computationally economical manner. The inputstereo signal is fed to MDCT transform modules 801, which output afrequency-domain representation of each channel. This is fed both to afinal switch 808 for actuating the encoder system between its predictionand direct coding modes and to a sum-and-difference stage 802. In directL/R coding or joint M/S coding—which is carried out in time frame forwhich the prediction coefficient α is set to zero—this embodimentsubjects the input signal to MDCT transformation, quantization andmultiplexing only, the latter two steps being effected by a combinedquantization and multiplexer stage 807 arranged at the output end of thesystem, where a bit stream is supplied. In prediction coding, each ofthe channels undergoes further processing between the sum-and-differencestage 802 and the switch 808. From the MDCT representation of the midsignal, the real-to-imaginary transform 804 derives MDST data andforwards these to both a prediction coefficient estimator 803 and aweighted summer 806. Like in the encoder systems shown in FIGS. 6 and 7,a further weighted summer 805 is used to combine the side signal withweighted MDCT and MDST representations of the mid signal to form aresidual channel signal, which is encoded together with the mid (i.e.,downmix) channel signal and the prediction coefficient by the combinedquantization and multiplexer module 807.

Turning now to FIG. 9, it will be illustrated that each of theembodiments of the encoder system may be combined with one or more TNS(analysis) filters. In accordance with the previous discussions, it isoften advantageous to apply TNS filtering to the signal in its downmixedform. Hence, as shown in FIG. 9, the adaptation of the encoder system ofFIG. 7 to include TNS is effected by adding TNS filters 911 immediatelyupstream of the combined quantization and multiplexer module 909.

Instead of the right/residual TNS filter 911 b, two separate TNS filters(not shown) may be provided immediately upstream of the portion of theswitch 910 adapted to handle the right or residual channel. Thus, eachof the two TNS filters will be supplied with the respective channelsignal data at all times, enabling TNS filtering based on more timeframes than the current one only. As has been already noted, TNS filtersare but one example of frequency-domain modifying devices, notablydevices basing their processing on more frame than the current one,which may benefit from such a placement as much as or more than at TNSfilter does.

As another possible alternative to the embodiment shown in FIG. 9, TNSfilters for selective activation can be arranged at more than one pointfor each channel. This is similar to the structure of the decoder systemshown in FIG. 4, where different sets of TNS filters can be connected bymeans of switches. This allows selection of the most suitable availablestage for TNS filtering for each time frame. In particular, it may beadvantageous to switch between different TNS locations in connectionwith switching between complex prediction stereo coding and other codingmodes.

FIG. 11 shows a variation based on the encoder system of FIG. 8, inwhich a second frequency-domain representation of the downmix signal isderived by means of a real-to-imaginary transform 1105. Similarly to thedecoder system shown in FIG. 4, this encoder system also includesselectively activable frequency-domain modifier modules, one 1102provided upstream of the downmix stage and one 1109 provided downstreamthereof. The frequency-domain modules 1102, 1109, which have in thisfigure been exemplified by TNS filters, can be connected into each ofthe signal paths using the four switches 1103 a, 1103 b, 1109 a and 1109b.

III. Non-Apparatus Embodiments

Embodiments of the third and a fourth aspects of the invention are shownin FIGS. 15 and 16. FIG. 15 shows a method for decoding a bit streaminto a stereo signal, comprising the following steps:

-   -   1. A bit stream is input.    -   2. The bit stream is dequantized, whereby a first        frequency-domain representation of downmix and residual channels        of a stereo signal are obtained.    -   3. A second frequency-domain representation of the downmix        channel is computed.    -   4. A side channel signal is computed on the basis of the three        frequency-domain representations of channels.    -   5. A stereo signal, preferably in left/right form, is computed        on the basis of the side and the downmix channels.    -   6. The stereo signal thus obtained is output.        Steps 3 through 5 may be regarded as a process of upmixing. Each        of steps 1 through 6 is analogous to the corresponding        functionality in any of the decoder systems disclosed in the        preceding portions of this text, and further details relating to        its implementation can be retrieved in the same portions.

FIG. 16 shows a method for encoding a stereo signal as a bit streamsignal, comprising the following steps:

-   -   1. A stereo signal is input.    -   2. The stereo signal is transformed into a first        frequency-domain representation.    -   3. A complex prediction coefficient is determined.    -   4. The frequency-domain representation is downmixed.    -   5. The downmix and residual channels are encoded as a bit stream        together with the complex prediction coefficient.    -   6. The bit stream is output.        Each of steps 1 through 5 is analogous to the corresponding        functionality in any of the encoder systems disclosed in the        preceding portions of this text, and further details relating to        its implementation can be retrieved in the same portions.

Both methods may be expressed as computer-readable instructions in theform of software programs and may be executed by a computer. The scopeof protection of this invention extends to such software andcomputer-program products for distributing such software.

IV. Empirical Evaluation

Several of the embodiments disclosed herein have been empiricallyassessed. The most important portions of the experimental materialobtained in this process will be summarized in this subsection.

The embodiment used for the experiments had the followingcharacteristics:

-   -   (i) Each MDST spectrum (for a time frame) was computed by        two-dimensional finite impulse response filtering from current,        previous and next MDCT spectra.    -   (ii) A psychoacoustic model from USAC stereo encoder was used.    -   (iii) The real and imaginary parts of the complex prediction        coefficient α were transmitted instead of the PS parameters ICC,        CLD and IPD. The real and imaginary parts were handled        independently, were limited to the range [−3.0, 3.0] and        quantized using a step size of 0.1. They were then        time-differentially coded and finally Huffman coded using the        scale factor codebook of the USAC. The prediction coefficients        were updated every second scale-factor band, which resulted in a        frequency resolution similar to that of MPEG Surround (see,        e.g., ISO/IEC 23003-1). This quantization and coding scheme        resulted in an average bit rate of approximately 2 kb/s for this        stereo side information in a typical configuration with a target        bit rate of 96 kb/s.    -   (iv) The bit stream format was modified without breaking current        USAC bit streams, as the 2-bit ms mask present bit stream        element currently has only three possible values. By using the        fourth value to indicate complex prediction allows for a        fallback mode of basic mid/side coding without any bits wasted        (for further details on this subject, see the previous        subsection of this disclosure).

The listening tests were accomplished according to the MUSHRAmethodology, entailing in particular playback over headphones and theuse of 8 test items with a sampling rate of 48 kHz. Three, five or sixtest subjects participated in each test.

The impact of different MDST approximations was evaluated to illustratethe practical complexity-to-quality trade-off that exists between theseoptions. The results are found in FIGS. 12 and 13, the former showingabsolute scores obtained and the latter showing differential scoresrelative to 96s USAC cp1f, that is, MDCT-domain unified stereo coding bycomplex prediction using a current MDCT frame to compute anapproximation of the MDST. It can be seen that the quality gain achievedby MDCT-based unified stereo coding increases when more computationallycomplex approaches to computing the MDST spectrum are applied.Considering the average over all test, the single-frame-based system 96sUSAC cp1f provides a significant increase in coding efficiency overconventional stereo coding. In turn, even significantly better resultsare obtained for 96s USAC cp3f, namely MDCT-domain unified stereo codingby complex prediction using current, previous and next MDCT frames tocompute the MDST.

V. Embodiments

Further, the invention may be embodied as a decoder system for decodinga bit stream signal into a stereo signal by complex prediction stereocoding, the decoder system comprising:

a dequantization stage (202; 401) for providing first frequency-domainrepresentations of a downmix signal (M) and a residual signal (D) basedon the bit stream signal, each of the first frequency-domainrepresentations comprising first spectral components representingspectral content of the corresponding signal expressed in a firstsubspace of a multidimensional space, wherein the first spectralcomponents are transform coefficients arranged in one or more timeframes of transform coefficients, each block generated by application ofa transform to a time segment of a time-domain signal; and

an upmix stage (206, 207, 210, 211; 406, 407, 408, 409) arrangeddownstream of the dequantization stage, adapted to generate the stereosignal based on the downmix signal and the residual signal andcomprising:

-   -   a module (206; 408) for computing a second frequency-domain        representation of the downmix signal based on the first        frequency-domain representation thereof, the second        frequency-domain representation comprising second spectral        components representing spectral content of the signal expressed        in a second subspace of the multidimensional space that includes        a portion of the multidimensional space not included in the        first subspace, said module being adapted to:        -   derive one or more first intermediate components from at            least some of the first spectral components;        -   form a combination of said one or more first spectral            components according to at least a portion of one or more            impulse responses to obtain one or more second intermediate            components; and        -   derive said one or more second spectral components from said            one or more second intermediate components;    -   a weighted summer (210, 211; 406, 407) for computing a side        signal (S) on the basis of the first and second frequency-domain        representations of the downmix signal, the first        frequency-domain representation of the residual signal and a        complex prediction coefficient (α) encoded in the bit stream        signal; and        -   a sum-and-difference stage (207; 409) for computing the            stereo signal on the basis of the first frequency-domain            representation of the downmix signal and the side signal.

Further still, the invention may be embodied as a decoder system fordecoding a bit stream signal into a stereo signal by complex predictionstereo coding, the decoder system comprising:

a dequantization stage (301) for providing first frequency-domainrepresentations of a downmix signal (M) and a residual signal (D) basedon the bit stream signal, each of the first frequency-domainrepresentations comprising first spectral components representingspectral content of the corresponding signal expressed in a firstsubspace of a multidimensional space; and

an upmix stage (306, 307, 308, 309, 312) arranged downstream of thedequantization stage, adapted to generate the stereo signal based on thedownmix signal and the residual signal and comprising:

-   -   a module (306, 307) for computing a second frequency-domain        representation of the downmix signal based on the first        frequency-domain representation thereof, the second        frequency-domain representation comprising second spectral        components representing spectral content of the signal expressed        in a second subspace of the multidimensional space that includes        a portion of the multidimensional space not included in the        first subspace, the module comprising:        -   an inverse transform stage (306) for computing a time-domain            representation of the downmix signal on the basis of the            first frequency-domain representation of the downmix signal            in the first subspace of the multidimensional space; and        -   a transform stage (307) for computing the second            frequency-domain representation of the downmix signal on the            basis of the time-domain representation of the signal;    -   a weighted summer (308, 309) for computing a side signal (S) on        the basis of the first and second frequency-domain        representations of the downmix signal, the first        frequency-domain representation of the residual signal and a        complex prediction coefficient (α) encoded in the bit stream        signal; and    -   a sum-and-difference stage (312) for computing the stereo signal        on the basis of the first frequency-domain representation of the        downmix signal and the side signal.

VI. Closing Remarks

Further embodiments of the present invention will become apparent to aperson skilled in the art after reading the description above. Eventhough the present description and drawings disclose embodiments andexamples, the invention is not restricted to these specific examples.Numerous modifications and variations can be made without departing fromthe scope of the present invention, which is defined by the accompanyingclaims.

It is noted that the methods and apparatus disclosed in this applicationmay be applied, after appropriate modifications within the skilledperson's abilities including routine experimentation, to coding ofsignals having more than two channels. It is particularly emphasizedthat any signals, parameters and matrices mentioned in connections withthe described embodiments may be frequency-variant orfrequency-invariant and/or time-variant or time-invariant. The describedcomputing steps may be carried out frequency-wise or for all frequencybands at a time, and all entities may be embodied to have afrequency-selective action. For the purposes of the application, anyquantization schemes may be adapted according to psycho-acoustic models.It is moreover noted that the various sum-and-difference conversions,that is, the conversion from downmix/residual form to pseudo-L/R form aswell as the L/R-to-M/S conversion and the M/S-to-L/R conversion, are allof the form

${g\begin{bmatrix}1 & 1 \\1 & {- 1}\end{bmatrix}},$

where, merely, the gain factor g may vary. Thus, by adjusting gainfactors individually, it is possible to compensate a certain encodinggain by an appropriate choice of decoding gain. Moreover, as the skilledperson realises, an even number of serially arranged sum-and-differencetransforms have the effect of a pass-through stage, possibly withnon-unity gain.

The systems and methods disclosed hereinabove may be implemented assoftware, firmware, hardware or a combination thereof. Certaincomponents or all components may be implemented as software executed bya digital signal processor or microprocessor, or be implemented ashardware or as an application-specific integrated circuit. Such softwaremay be distributed on computer readable media, which may comprisecomputer storage media and communication media. As is well known to aperson skilled in the art, computer storage media includes both volatileand non-volatile, removable and non-removable media implemented in anymethod or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium which can be used to store the desired informationand which can be accessed by a computer. Further, it is known to theskilled person that communication media typically embodies computerreadable instructions, data structures, program modules or other data ina modulated data signal such as a carrier wave or other transportmechanism and includes any information delivery media.

1. An apparatus for outputting a stereo audio signal having a leftchannel and a right channel, the apparatus comprising: a demultiplexerconfigured to receive an audio bitstream and extract therefrom aprediction coefficient for each of a plurality of frequency bands; adecoder configured to generate a downmix signal and a residual signalfrom the encoded audio bitstream; and an upmixer configured to operatein either a prediction mode or a non-prediction mode based on aparameter encoded in the audio bitstream, and to output the left channeland the right channel as the stereo audio signal, wherein, when theupmixer operates in the prediction mode, the upmixer generates the leftchannel and the right channel from a combination of the downmix signal,the residual signal, and the prediction coefficients for each of theplurality of frequency bands, and wherein, when the upmixer operates inthe non-prediction mode, the upmixer generates the left channel based ona sum of the downmix signal and the residual signal passed through fromthe decoder, and generates the right channel based on a difference ofthe downmix signal and the residual signal passed through from thedecoder.
 2. The apparatus of claim 1 wherein the parameter is theprediction coefficient.
 3. The apparatus of claim 2 wherein the upmixeroperates in the non-prediction mode when a value of the predictioncoefficient equals zero or is smaller than a predetermined tolerance andthe upmixer operates in the prediction mode for all other values of theprediction coefficient.
 4. The apparatus of claim 1 wherein theprediction coefficient reduces or minimizes an energy of the residualsignal.
 5. The apparatus of claim 1 further comprising a noise shaperconfigured to shape a noise associated with the downmix signal, whereinthe noise shaper is arranged upstream of the upmixer.
 6. The apparatusof claim 5 wherein the noise shaper is a temporal noise shaperconfigured to shape the noise over time.
 7. The apparatus of claim 1wherein, when the upmixer operates in the prediction mode, the upmixergenerates the left channel and the right channel using a filter havingthree taps.
 8. The apparatus of claim 1 wherein the downmix signalcomprises a mid signal formed by a linear combination of an originalleft channel and an original right channel.
 9. The apparatus of claim 1wherein the prediction coefficient is a real valued coefficient.
 10. Theapparatus of claim 1 wherein the prediction coefficient is a complexvalued coefficient.
 11. The apparatus of claim 1 wherein the upmixercombines the side signal with the downmix signal by adding a version ofthe downmix signal with a version of the side signal to generate theleft channel and by subtracting a version of the side signal from aversion of the downmix signal to generate the right channel.
 12. Theapparatus of claim 1 wherein the prediction coefficient is coded in theaudio bitstream and the demultiplexer is further configured to decodethe prediction coefficient.
 13. The apparatus of claim 1 wherein theupmixer is further configured to add the residual signal to the sidesignal when operating in the prediction mode.
 14. A method foroutputting a stereo audio signal having a left channel and a rightchannel, the method comprising: receiving an audio bitstream and extracttherefrom a prediction coefficient for each of a plurality of frequencybands; generating in a decoder a downmix signal and a residual signalfrom the encoded audio bitstream; upmixing in either a prediction modeor a non-prediction mode based on a parameter encoded in the audiobitstream; and outputting the left channel and the right channel as thestereo audio signal, wherein, when the upmixing operates in theprediction mode, the upmixing generates the left channel and the rightchannel from a combination of the downmix signal, the residual signal,and the prediction coefficients for each of the plurality of frequencybands, and wherein, when the upmixing operates in the non-predictionmode, the upmixing generates the left channel based on a sum of thedownmix signal and the residual signal passed through from the decoder,and generates the right channel based on a difference of the downmixsignal and the residual signal passed through from the decoder.
 15. Acomputer-readable medium containing instructions that when executed by aprocessor perform the method of claim 14.